Manticore Search accepts the USR1 signal for reopening server and query log files.
The official DEB and RPM packages install a Logrotate configuration file for all files in the default log folder.
A simple logrotate configuration for log files looks like:
/var/log/manticore/*.log {
weekly
rotate 10
copytruncate
delaycompress
compress
notifempty
missingok
}mysql> FLUSH LOGS;
Query OK, 0 rows affected (0.01 sec)Additionally, the FLUSH LOGS SQL command is available, which works the same way as the USR1 system signal. It initiates the reopening of searchd log and query log files, allowing you to implement log file rotation. The command is non-blocking (i.e., it returns immediately).
⪢ Node info and management
The easiest way to view high-level information about your Manticore node is by running status in the MySQL client. It will display information about various aspects, such as:
- Current version
- Whether SSL is in effect or not
- Current TCP port/Unix socket
- Uptime
- Number of threads
- Number of jobs in queue
- Number of connections (
clients) - Number of tasks being processed currently
- Number of queries executed since the start
- Number of jobs in queue and number of tasks, normalized by the number of threads
- SQL
mysql> status--------------
mysql Ver 14.14 Distrib 5.7.30, for Linux (x86_64) using EditLine wrapper
Connection id: 378
Current database: Manticore
Current user: Usual
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 3.4.3 a48c61d6@200702 coroutines git branch coroutines_work_junk...origin/coroutines_work_junk
Protocol version: 10
Connection: 0 via TCP/IP
Server characterset:
Db characterset:
Client characterset: utf8
Conn. characterset: utf8
TCP port: 8306
Uptime: 23 hours 6 sec
Threads: 12 Queue: 3 Clients: 1 Vip clients: 0 Tasks: 5 Queries: 318967 Wall: 7h CPU: 0us
Queue/Th: 0.2 Tasks/Th: 0.4
--------------SHOW STATUS [ LIKE pattern ]SHOW STATUS is an SQL statement that presents various helpful performance counters. IO and CPU counters will only be available if searchd was started with the --iostats and --cpustats switches, respectively (or if they were enabled via SET GLOBAL iostats/cpustats=1).
- SQL
SHOW STATUS;+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Counter | Value |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| uptime | 182 |
| connections | 368 |
| maxed_out | 0 |
| version | 6.3.7 b10359434@24100213 dev (columnar 2.3.1 bd59d08@24093015) (secondary 2.3.1 bd59d08@24093015) (knn 2.3.1 bd59d08@24093015) (buddy v2.3.13) |
| mysql_version | 6.3.7 b10359434@24100213 dev (columnar 2.3.1 bd59d08@24093015) (secondary 2.3.1 bd59d08@24093015) (knn 2.3.1 bd59d08@24093015) |
| command_search | 4 |
| command_excerpt | 0 |
| command_update | 2 |
| command_keywords | 0 |
| command_persist | 0 |
| command_status | 8 |
| command_flushattrs | 0 |
| command_sphinxql | 0 |
| command_ping | 0 |
| command_delete | 3 |
| command_set | 0 |
| command_insert | 3 |
| command_replace | 3 |
| command_commit | 0 |
| command_suggest | 0 |
| command_json | 0 |
| command_callpq | 0 |
| command_cluster | 0 |
| command_getfield | 0 |
| insert_replace_stats_ms_avg | 42.625 35.895 35.895 |
| insert_replace_stats_ms_min | 0.096 0.096 0.096 |
| insert_replace_stats_ms_max | 85.154 85.154 85.154 |
| insert_replace_stats_ms_pct95 | 42.625 42.625 42.625 |
| insert_replace_stats_ms_pct99 | 42.625 42.625 42.625 |
| search_stats_ms_avg | 0.209 0.188 0.188 |
| search_stats_ms_min | 0.205 0.058 0.058 |
| search_stats_ms_max | 0.214 0.278 0.278 |
| search_stats_ms_pct95 | 0.214 0.278 0.278 |
| search_stats_ms_pct99 | 0.214 0.278 0.278 |
| update_stats_ms_avg | 0.024 0.024 0.024 |
| update_stats_ms_min | 0.007 0.007 0.007 |
| update_stats_ms_max | 0.042 0.042 0.042 |
| update_stats_ms_pct95 | 0.042 0.042 0.042 |
| update_stats_ms_pct99 | 0.042 0.042 0.042 |
| agent_connect | 0 |
| agent_tfo | 0 |
| agent_retry | 0 |
| queries | 6 |
| dist_queries | 0 |
| workers_total | 32 |
| workers_active | 3 |
| workers_clients | 1 |
| workers_clients_vip | 0 |
| workers_clients_buddy | 1 |
| work_queue_length | 6 |
| load | 0.03 0.03 0.03 |
| load_primary | 0.00 0.00 0.00 |
| load_secondary | 0.00 0.00 0.00 |
| query_wall | 0.000 |
| query_cpu | OFF |
| dist_wall | 0.000 |
| dist_local | 0.000 |
| dist_wait | 0.000 |
| query_reads | OFF |
| query_readkb | OFF |
| query_readtime | OFF |
| avg_query_wall | 0.000 |
| avg_query_cpu | OFF |
| avg_dist_wall | 0.000 |
| avg_dist_local | 0.000 |
| avg_dist_wait | 0.000 |
| avg_query_reads | OFF |
| avg_query_readkb | OFF |
| avg_query_readtime | OFF |
| qcache_max_bytes | 16777216 |
| qcache_thresh_msec | 3000 |
| qcache_ttl_sec | 60 |
| qcache_cached_queries | 0 |
| qcache_used_bytes | 0 |
| qcache_hits | 0 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+An optional LIKE clause is supported, allowing you to select only the variables that match a specific pattern. The pattern syntax follows standard SQL wildcards, where % represents any number of any characters, and _ represents a single character.
- qcache
- perf_stats
SHOW STATUS LIKE 'qcache%';+-----------------------+-------+
| Counter | Value |
+-----------------------+-------+
| qcache_max_bytes | 0 |
| qcache_thresh_msec | 3000 |
| qcache_ttl_sec | 60 |
| qcache_cached_queries | 0 |
| qcache_used_bytes | 0 |
| qcache_hits | 0 |
+-----------------------+-------+The SHOW STATUS command gives a detailed report on various performance metrics in Manticore, including query time statistics for insert/replace, search, and update queries. These stats are calculated over sliding windows of 1, 5, and 15 minutes, showing average, minimum, maximum, and 95th/99th percentile values for query times.
These metrics help track performance over specific time intervals, making it easier to spot trends in query response times and find possible bottlenecks.
The following metrics are part of the SHOW STATUS output:
*_avg: The average query time for each type of query over the last 1, 5, and 15 minutes.*_min: The shortest query time recorded for each query type.*_max: The longest query time recorded for each query type.*_pct95: The time under which 95% of queries are completed.*_pct99: The time under which 99% of queries are completed.
These statistics are provided separately for insert/replace (insert_replace_stats_*), search (search_stats_*), and update (update_stats_*) queries, offering detailed insights into the performance of different operations.
If no queries are executed during the monitored interval, the system will display N/A.
- perf_stats
SHOW STATUS LIKE '%stats_ms%';+-------------------------------+-------------------+
| Counter | Value |
+-------------------------------+-------------------+
| insert_replace_stats_ms_avg | N/A 35.895 35.895 |
| insert_replace_stats_ms_min | N/A 0.096 0.096 |
| insert_replace_stats_ms_max | N/A 85.154 85.154 |
| insert_replace_stats_ms_pct95 | N/A 42.625 42.625 |
| insert_replace_stats_ms_pct99 | N/A 42.625 42.625 |
| search_stats_ms_avg | N/A 0.188 0.188 |
| search_stats_ms_min | N/A 0.058 0.058 |
| search_stats_ms_max | N/A 0.278 0.278 |
| search_stats_ms_pct95 | N/A 0.278 0.278 |
| search_stats_ms_pct99 | N/A 0.278 0.278 |
| update_stats_ms_avg | N/A 0.024 0.024 |
| update_stats_ms_min | N/A 0.007 0.007 |
| update_stats_ms_max | N/A 0.042 0.042 |
| update_stats_ms_pct95 | N/A 0.042 0.042 |
| update_stats_ms_pct99 | N/A 0.042 0.042 |
+-------------------------------+-------------------+SHOW SETTINGS is an SQL statement that displays the current settings from your configuration file. The setting names are represented in the following format: 'config_section_name'.'setting_name'
The result also includes two additional values:
configuration_file- The path to the configuration fileworker_pid- The process ID of the runningsearchdinstance
- SQL
SHOW SETTINGS;+--------------------------+-------------------------------------+
| Setting_name | Value |
+--------------------------+-------------------------------------+
| configuration_file | /etc/manticoresearch/manticore.conf |
| worker_pid | 658 |
| searchd.listen | 0.0.0:9312 |
| searchd.listen | 0.0.0:9306:mysql |
| searchd.listen | 0.0.0:9308:http |
| searchd.log | /var/log/manticore/searchd.log |
| searchd.query_log | /var/log/manticore/query.log |
| searchd.pid_file | /var/run/manticore/searchd.pid |
| searchd.data_dir | /var/lib/manticore |
| searchd.query_log_format | sphinxql |
| searchd.binlog_path | /var/lib/manticore/binlog |
| common.plugin_dir | /usr/local/lib/manticore |
| common.lemmatizer_base | /usr/share/manticore/morph/ |
+--------------------------+-------------------------------------+
13 rows in set (0.00 sec)SHOW AGENT ['agent_or_index'] STATUS [ LIKE pattern ]SHOW AGENT STATUS displays the statistics of remote agents or a distributed table. It includes values such as the age of the last request, last answer, the number of various types of errors and successes, and so on. Statistics are displayed for every agent for the last 1, 5, and 15 intervals, each consisting of ha_period_karma seconds.
- SQL
- PHP
- Python
- javascript
- Java
- C#
- TypeScript
- Go
SHOW AGENT STATUS;+------------------------------------+----------------------------+
| Variable_name | Value |
+------------------------------------+----------------------------+
| status_period_seconds | 60 |
| status_stored_periods | 15 |
| ag_0_hostname | 192.168.0.202:6713 |
| ag_0_references | 2 |
| ag_0_lastquery | 0.41 |
| ag_0_lastanswer | 0.19 |
| ag_0_lastperiodmsec | 222 |
| ag_0_pingtripmsec | 10.521 |
| ag_0_errorsarow | 0 |
| ag_0_1periods_query_timeouts | 0 |
| ag_0_1periods_connect_timeouts | 0 |
| ag_0_1periods_connect_failures | 0 |
| ag_0_1periods_network_errors | 0 |
| ag_0_1periods_wrong_replies | 0 |
| ag_0_1periods_unexpected_closings | 0 |
| ag_0_1periods_warnings | 0 |
| ag_0_1periods_succeeded_queries | 27 |
| ag_0_1periods_msecsperquery | 232.31 |
| ag_0_5periods_query_timeouts | 0 |
| ag_0_5periods_connect_timeouts | 0 |
| ag_0_5periods_connect_failures | 0 |
| ag_0_5periods_network_errors | 0 |
| ag_0_5periods_wrong_replies | 0 |
| ag_0_5periods_unexpected_closings | 0 |
| ag_0_5periods_warnings | 0 |
| ag_0_5periods_succeeded_queries | 146 |
| ag_0_5periods_msecsperquery | 231.83 |
| ag_1_hostname | 192.168.0.202:6714 |
| ag_1_references | 2 |
| ag_1_lastquery | 0.41 |
| ag_1_lastanswer | 0.19 |
| ag_1_lastperiodmsec | 220 |
| ag_1_pingtripmsec | 10.004 |
| ag_1_errorsarow | 0 |
| ag_1_1periods_query_timeouts | 0 |
| ag_1_1periods_connect_timeouts | 0 |
| ag_1_1periods_connect_failures | 0 |
| ag_1_1periods_network_errors | 0 |
| ag_1_1periods_wrong_replies | 0 |
| ag_1_1periods_unexpected_closings | 0 |
| ag_1_1periods_warnings | 0 |
| ag_1_1periods_succeeded_queries | 27 |
| ag_1_1periods_msecsperquery | 231.24 |
| ag_1_5periods_query_timeouts | 0 |
| ag_1_5periods_connect_timeouts | 0 |
| ag_1_5periods_connect_failures | 0 |
| ag_1_5periods_network_errors | 0 |
| ag_1_5periods_wrong_replies | 0 |
| ag_1_5periods_unexpected_closings | 0 |
| ag_1_5periods_warnings | 0 |
| ag_1_5periods_succeeded_queries | 146 |
| ag_1_5periods_msecsperquery | 230.85 |
+------------------------------------+----------------------------+
50 rows in set (0.01 sec)An optional LIKE clause is supported, with the syntax being the same as in SHOW STATUS.
- SQL
- PHP
- Python
- javascript
- Java
- C#
- TypeScript
- Go
SHOW AGENT STATUS LIKE '%5period%msec%';+-----------------------------+--------+
| Key | Value |
+-----------------------------+--------+
| ag_0_5periods_msecsperquery | 234.72 |
| ag_1_5periods_msecsperquery | 233.73 |
| ag_2_5periods_msecsperquery | 343.81 |
+-----------------------------+--------+
3 rows in set (0.00 sec)You can specify a particular agent by its address. In this case, only that agent's data will be displayed. Additionally, the agent_ prefix will be used instead of ag_N_:
- SQL
- PHP
- Python
- javascript
- Java
- C#
- TypeScript
- Go
SHOW AGENT '192.168.0.202:6714' STATUS LIKE '%15periods%';+-------------------------------------+--------+
| Variable_name | Value |
+-------------------------------------+--------+
| agent_15periods_query_timeouts | 0 |
| agent_15periods_connect_timeouts | 0 |
| agent_15periods_connect_failures | 0 |
| agent_15periods_network_errors | 0 |
| agent_15periods_wrong_replies | 0 |
| agent_15periods_unexpected_closings | 0 |
| agent_15periods_warnings | 0 |
| agent_15periods_succeeded_queries | 439 |
| agent_15periods_msecsperquery | 231.73 |
+-------------------------------------+--------+
9 rows in set (0.00 sec)Finally, you can check the status of the agents in a specific distributed table using the SHOW AGENT table_name STATUS statement. This statement displays the table's HA status (i.e., whether or not it uses agent mirrors at all) and provides information on the mirrors, including: address, blackhole and persistent flags, and the mirror selection probability used when one of the weighted probability strategies is in effect.
- SQL
- PHP
- Python
- javascript
- Java
- C#
- TypeScript
- Go
SHOW AGENT dist_index STATUS;+--------------------------------------+--------------------------------+
| Variable_name | Value |
+--------------------------------------+--------------------------------+
| dstindex_1_is_ha | 1 |
| dstindex_1mirror1_id | 192.168.0.202:6713:loc |
| dstindex_1mirror1_probability_weight | 0.372864 |
| dstindex_1mirror1_is_blackhole | 0 |
| dstindex_1mirror1_is_persistent | 0 |
| dstindex_1mirror2_id | 192.168.0.202:6714:loc |
| dstindex_1mirror2_probability_weight | 0.374635 |
| dstindex_1mirror2_is_blackhole | 0 |
| dstindex_1mirror2_is_persistent | 0 |
| dstindex_1mirror3_id | dev1.manticoresearch.com:6714:loc |
| dstindex_1mirror3_probability_weight | 0.252501 |
| dstindex_1mirror3_is_blackhole | 0 |
| dstindex_1mirror3_is_persistent | 0 |
+--------------------------------------+--------------------------------+
13 rows in set (0.00 sec)